SHOW TABLE STATUS is an SQL statement that displays various per-table statistics.

The syntax is:

SHOW TABLE table_name STATUS

Depending on index type, displayed statistic includes different set of rows:

• template: index_type.
• distributed: index_type, query_time_1min, query_time_5min,query_time_15min,query_time_total, exact_query_time_1min, exact_query_time_5min, exact_query_time_15min, exact_query_time_total, found_rows_1min, found_rows_5min, found_rows_15min, found_rows_total.
• percolate: index_type, stored_queries, ram_bytes, disk_bytes, max_stack_need, average_stack_base, desired_thread_stack, tid, tid_saved, query_time_1min, query_time_5min,query_time_15min,query_time_total, exact_query_time_1min, exact_query_time_5min, exact_query_time_15min, exact_query_time_total, found_rows_1min, found_rows_5min, found_rows_15min, found_rows_total.
• plain: index_type, indexed_documents, indexed_bytes, may be set of field_tokens_* and total_tokens, ram_bytes, disk_bytes, disk_mapped, disk_mapped_cached, disk_mapped_doclists, disk_mapped_cached_doclists, disk_mapped_hitlists, disk_mapped_cached_hitlists, killed_documents, killed_rate, query_time_1min, query_time_5min,query_time_15min,query_time_total, exact_query_time_1min, exact_query_time_5min, exact_query_time_15min, exact_query_time_total, found_rows_1min, found_rows_5min, found_rows_15min, found_rows_total.
• rt: index_type, indexed_documents, indexed_bytes, may be set of field_tokens_* and total_tokens, ram_bytes, disk_bytes, disk_mapped, disk_mapped_cached, disk_mapped_doclists, disk_mapped_cached_doclists, disk_mapped_hitlists, disk_mapped_cached_hitlists, killed_documents, killed_rate, ram_chunk, ram_chunk_segments_count, disk_chunks, mem_limit, mem_limit_rate, ram_bytes_retired, optimizing, locked, tid, tid_saved, query_time_1min, query_time_5min,query_time_15min,query_time_total, exact_query_time_1min, exact_query_time_5min, exact_query_time_15min, exact_query_time_total, found_rows_1min, found_rows_5min, found_rows_15min, found_rows_total.

Here is the meaning of these values:

• index_type: currently one of disk, rt, percolate, template, and distributed.
• indexed_documents: number of indexed documents.
• indexed_bytes: overall size of indexed text. Notice, this value is not strict, since in full-text index that is impossible to strictly recover back stored text to measure it.
• stored_queries: number of percolate queries, stored in the table.
• field_tokens_XXX: optional, total per-field lengths (in tokens) across the entire table (used internally for BM25A and BM25F ranking functions). Only available for tables built with index_field_lengths=1.
• total_tokens: optional, overall sum of all field_tokens_XXX.
• ram_bytes: total RAM occupied by table.
• disk_bytes: total disk space, occupied by table.
• disk_mapped: total size of file mappings.
• disk_mapped_cached: total size of file mappings actually cached in RAM.
• disk_mapped_doclists and disk_mapped_cached_doclists: portion of total and cached mappings belonging to document lists.
• disk_mapped_hitlists and disk_mapped_cached_hitlists: portion of total and cached mappings belonging to hit lists. Doclists and hitlists values are shown separately since they're typically large (e.g., about 90% of the whole table's size).
• killed_documents and killed_rate: the first indicates the number of deleted documents and the rate of deleted/indexed. Technically, deleting a document means suppressing it in search output, but it still physically exists in the table and will only be purged after merging/optimizing the table.
• ram_chunk: size of the RAM chunk of real-time or percolate table.
• ram_chunk_segments_count: RAM chunk is internally composed of segments, typically no more than 32. This line shows the current count.
• disk_chunks: number of disk chunks in the real-time table.
• mem_limit: actual value of rt_mem_limit for the table.
• mem_limit_rate: the rate at which the RAM chunk will be flushed as a disk chunk, e.g., if rt_mem_limit is 128M and the rate is 50%, a new disk chunk will be saved when the RAM chunk exceeds 64M.
• ram_bytes_retired: represents the size of garbage in RAM chunks (e.g., deleted or replaced documents not yet permanently removed).
• optimizing: a value greater than 0 indicates that the table is currently performing optimization (i.e. it is merging some disk chunks right now).
• locked: a value greater than 0 indicates that the table is currently locked by FREEZE. The number represents how many times the table has been frozen. For instance, a table might be frozen by manticore-backup and then frozen again by replication. It should only be completely unfrozen when no other process requires it to be frozen.
• max_stack_need: stack space we need to calculate most complex from the stored percolate queries. That is dynamic value, depends on build details as compiler, optimization, hardware, etc.
• average_stack_base: stack space which is usually occupied on start of calculation of percolate query.
• desired_thread_stack: sum of above values, rounded up to 128 bytes edge. If this value is greater than thread_stack, you may not execute call pq over this table, as some stored queries will fail. Default thread_stack value is 1M (which is 1048576); other values should be configured.
• tid and tid_saved: represent the state of saving the table. tid increases with each change (transaction). tid_saved shows the max tid of the state saved in a RAM chunk in <table>.ram file. When the numbers differ, some changes exist only in RAM and are also backed by binlog (if enabled). Performing FLUSH TABLE or scheduling periodic flushing saves these changes. After flushing, the binlog is cleared, and tid_saved represents the new actual state.
• query_time_*, exact_query_time_*: query execution time statistics for the last 1 minute, 5 minutes, 15 minutes, and total since server start; data is encapsulated as a JSON object, including the number of queries and min, max, avg, 95, and 99 percentile values.
• found_rows_*: statistics of rows found by queries; provided for the last 1 minute, 5 minutes, 15 minutes, and total since server start; data is encapsulated as a JSON object, including the number of queries and min, max, avg, 95, and 99 percentile values.

SHOW TABLE SETTINGS is an SQL statement that displays per-table settings in a format compatible with the config file.

The syntax is:

SHOW TABLE table_name[.N | CHUNK N] SETTINGS

The output resembles the --dumpconfig option of the indextool utility. The report provides a breakdown of all table settings, including tokenizer and dictionary options.

You can also specify a particular chunk number to view the settings of a specific chunk in an RT table. The numbering is 0-based.

When using Update to modify document attributes in real-time, the changes are first written to an in-memory copy of the attributes. These updates occur in a memory-mapped file, meaning the OS decides when to write the changes to disk. Upon normal shutdown of searchd (triggered by a SIGTERM signal), all changes are forced to be written to disk.

You can also instruct searchd to periodically write these changes back to disk to prevent data loss. The interval between these flushes is determined by attr_flush_period, specified in seconds (or special_suffixes).

By default, the value is 0, which disables periodic flushing. However, flushing will still occur during a normal shutdown.

This setting controls the automatic OPTIMIZE process for table compaction.

By default table compaction occurs automatically. You can modify this behavior with the auto_optimize setting:

• 0 to disable automatic table compaction (you can still call OPTIMIZE manually)
• 1 to explicitly enable it
• to enable it while multiplying the optimization threshold by 2.

By default, OPTIMIZE runs until the number of disk chunks is less than or equal to the number of logical CPU cores multiplied by 2.

However, if the table has attributes with KNN indexes, this threshold is different. In this case, it is set to the number of physical CPU cores divided by 2 to improve KNN search performance.

Note that toggling auto_optimize on or off doesn't prevent you from running OPTIMIZE TABLE manually.

Manticore supports the automatic creation of tables that don't yet exist but are specified in INSERT statements. This feature is enabled by default. To disable it, set auto_schema = 0 explicitly in your configuration. To re-enable it, set auto_schema = 1 or remove the auto_schema setting from the configuration.

Keep in mind that the /bulk HTTP endpoint does not support automatic table creation.

NOTE: The auto schema functionality requires Manticore Buddy. If it doesn't work, make sure Buddy is installed.

This setting controls the binary log transaction flush/sync mode. It is optional, with a default value of 2 (flush every transaction, sync every second).

The directive determines how frequently the binary log will be flushed to the OS and synced to disk. There are three supported modes:

• 0, flush and sync every second. This offers the best performance, but up to 1 second worth of committed transactions can be lost in the event of a server crash or an OS/hardware crash.
• 1, flush and sync every transaction. This mode provides the worst performance but guarantees that every committed transaction's data is saved.
• 2, flush every transaction, sync every second. This mode delivers good performance and ensures that every committed transaction is saved in case of a server crash. However, in the event of an OS/hardware crash, up to 1 second worth of committed transactions can be lost.

For those familiar with MySQL and InnoDB, this directive is similar to innodb_flush_log_at_trx_commit. In most cases, the default hybrid mode 2 provides a nice balance of speed and safety, with full RT table data protection against server crashes and some protection against hardware ones.

This setting controls the maximum binary log file size. It is optional, with a default value of 256 MB.

A new binlog file will be forcibly opened once the current binlog file reaches this size limit. This results in a finer granularity of logs and can lead to more efficient binlog disk usage under certain borderline workloads. A value of 0 indicates that the binlog file should not be reopened based on size.

This setting determines the path for binary log (also known as transaction log) files. It is optional, with a default value of the build-time configured data directory (e.g., /var/lib/manticore/data/binlog.* in Linux).

Binary logs are used for crash recovery of RT table data and for attribute updates of plain disk indices that would otherwise only be stored in RAM until flush. When logging is enabled, every transaction COMMIT-ted into an RT table is written into a log file. Logs are then automatically replayed on startup after an unclean shutdown, recovering the logged changes.

The binlog_path directive specifies the location of binary log files. It should only contain the path; searchd will create and unlink multiple binlog.* files in the directory as necessary (including binlog data, metadata, and lock files, etc).

An empty value disables binary logging, which improves performance but puts the RT table data at risk.

This setting determines the path to the Manticore Buddy binary. It is optional, with a default value being the build-time configured path, which varies across different operating systems. Typically, you don't need to modify this setting. However, it may be useful if you wish to run Manticore Buddy in debug mode, make changes to Manticore Buddy, or implement a new plugin. In the latter case, you can git clone Buddy from https://github.com/manticoresoftware/manticoresearch-buddy, add a new plugin to the directory ./plugins/, and run composer install --prefer-source for easier development after you change the directory to the Buddy source.

To ensure you can run composer, your machine must have PHP 8.2 or higher installed with the following extensions:

--enable-dom
--with-libxml
--enable-tokenizer
--enable-xml
--enable-xmlwriter
--enable-xmlreader
--enable-simplexml
--enable-phar
--enable-bcmath
--with-gmp
--enable-debug
--with-mysqli
--enable-mysqlnd

You can also opt for the special manticore-executor-dev version for Linux amd64 available in the releases, for example: https://github.com/manticoresoftware/executor/releases/tag/v1.0.13

If you go this route, remember to link the dev version of the manticore executor to /usr/bin/php.

To disable Manticore Buddy, set the value to empty as shown in the example.

This setting determines the maximum time to wait between requests (in seconds or special_suffixes) when using persistent connections. It is optional, with a default value of five minutes.

Server libc locale. Optional, default is C.

Specifies the libc locale, affecting the libc-based collations. Refer to collations section for the details.

Default server collation. Optional, default is libc_ci.

Specifies the default collation used for incoming requests. The collation can be overridden on a per-query basis. Refer to collations section for the list of available collations and other details.

When specified, this setting enables the real-time mode, which is an imperative way of managing data schema. The value should be a path to the directory where you want to store all your tables, binary logs, and everything else needed for the proper functioning of Manticore Search in this mode. Indexing of plain tables is not allowed when the data_dir is specified. Read more about the difference between the RT mode and the plain mode in this section.

The timeout for preventing auto-flushing a RAM chunk if there are no searches in the table. Optional, default is 30 seconds.

The time to check for searches before determining whether to auto-flush. Auto-flushing will occur only if there has been at least one search in the table within the last diskchunk_flush_search_timeout seconds. Works in conjunction with diskchunk_flush_write_timeout. The corresponding per-table setting has a higher priority and will override this instance-wide default, providing more fine-grained control.

The time in seconds to wait without a write before auto-flushing the RAM chunk to disk. Optional, default is 1 second.

If no write occurs in the RAM chunk within diskchunk_flush_write_timeout seconds, the chunk will be flushed to disk. Works in conjunction with diskchunk_flush_search_timeout. To disable auto-flush, set diskchunk_flush_write_timeout = -1 explicitly in your configuration. The corresponding per-table setting has a higher priority and will override this instance-wide default, providing more fine-grained control.

This setting specifies the maximum size of document blocks from document storage that are held in memory. It is optional, with a default value of 16m (16 megabytes).

When stored_fields is used, document blocks are read from disk and uncompressed. Since every block typically holds several documents, it may be reused when processing the next document. For this purpose, the block is held in a server-wide cache. The cache holds uncompressed blocks.

Default attribute storage engine used when creating tables in RT mode. Can be rowwise (default) or columnar.

This setting determines the maximum number of expanded keywords for a single wildcard. It is optional, with a default value of 0 (no limit).

When performing substring searches against tables built with dict = keywords enabled, a single wildcard may potentially result in thousands or even millions of matched keywords (think of matching a* against the entire Oxford dictionary). This directive allows you to limit the impact of such expansions. Setting expansion_limit = N restricts expansions to no more than N of the most frequent matching keywords (per each wildcard in the query).

This setting determines the maximum number of documents in the expanded keyword that allows merging all such keywords together. It is optional, with a default value of 32.

When performing substring searches against tables built with dict = keywords enabled, a single wildcard may potentially result in thousands or even millions of matched keywords. This directive allows you to increase the limit of how many keywords will merge together to speed up matching but uses more memory in the search.

This setting determines the maximum number of hits in the expanded keyword that allows merging all such keywords together. It is optional, with a default value of 256.

When performing substring searches against tables built with dict = keywords enabled, a single wildcard may potentially result in thousands or even millions of matched keywords. This directive allows you to increase the limit of how many keywords will merge together to speed up matching but uses more memory in the search.

This setting specifies the agent mirror statistics window size, in seconds (or special_suffixes). It is optional, with a default value of 60 seconds.

For a distributed table with agent mirrors in it (see more in agent, the master tracks several different per-mirror counters. These counters are then used for failover and balancing (the master picks the best mirror to use based on the counters). Counters are accumulated in blocks of ha_period_karma seconds.

After beginning a new block, the master may still use the accumulated values from the previous one until the new one is half full. As a result, any previous history stops affecting the mirror choice after 1.5 times ha_period_karma seconds at most.

Even though at most two blocks are used for mirror selection, up to 15 last blocks are stored for instrumentation purposes. These blocks can be inspected using the SHOW AGENT STATUS statement.

This setting configures the interval between agent mirror pings, in milliseconds (or special_suffixes). It is optional, with a default value of 1000 milliseconds.

For a distributed table with agent mirrors in it (see more in agent), the master sends all mirrors a ping command during idle periods. This is to track the current agent status (alive or dead, network roundtrip, etc). The interval between such pings is defined by this directive. To disable pings, set ha_ping_interval to 0.

The listen_backlog setting determines the length of the TCP listen backlog for incoming connections. This is particularly relevant for Windows builds that process requests one by one. When the connection queue reaches its limit, new incoming connections will be refused. For non-Windows builds, the default value should work fine, and there is usually no need to adjust this setting.

This setting lets you specify an IP address and port, or Unix-domain socket path, that Manticore will accept connections on.

The general syntax for listen is:

listen = ( address ":" port | port | path | address ":" port start - port end ) [ ":" protocol [ "_vip" ] [ "_readonly" ] ]

You can specify:

• either an IP address (or hostname) and a port number
• or just a port number
• or a Unix socket path (not supported on Windows)
• or an IP address and port range

If you specify a port number but not an address, searchd will listen on all network interfaces. Unix path is identified by a leading slash. Port range can be set only for the replication protocol.

You can also specify a protocol handler (listener) to be used for connections on this socket. The listeners are:

• Not specified - Manticore will accept connections at this port from:
  • other Manticore agents (i.e., a remote distributed table)
  • clients via HTTP and HTTPS
  • Manticore Buddy. Ensure you have a listener of this kind (or an http listener, as mentioned below) to avoid limitations in Manticore functionality.
• mysql MySQL protocol for connections from MySQL clients. Note:
  • Compressed protocol is also supported.
  • If SSL is enabled, you can make an encrypted connection.
• replication - replication protocol used for nodes communication. More details can be found in the replication section. You can specify multiple replication listeners, but they must all listen on the same IP; only the ports can be different. When you define a replication listener with a port range (e.g., listen = 192.168.0.1:9320-9328:replication), Manticore doesn't immediately start listening on these ports. Instead, it will take random free ports from the specified range only when you start using replication. At least 2 ports are required in the range for replication to work properly.
• http - same as Not specified. Manticore will accept connections at this port from remote agents and clients via HTTP and HTTPS.
• https - HTTPS protocol. Manticore will accept only HTTPS connections at this port. More details can be found in section SSL.
• sphinx - legacy binary protocol. Used to serve connections from remote SphinxSE clients. Some Sphinx API clients implementations (an example is the Java one) require the explicit declaration of the listener.

Adding suffix _vip to client protocols (that is, all except replication, for instance mysql_vip or http_vip or just _vip) forces creating a dedicated thread for the connection to bypass different limitations. That's useful for node maintenance in case of severe overload when the server would either stall or not let you connect via a regular port otherwise.

Suffix _readonly sets read-only mode for the listener and limits it to accept only read queries.

The log setting specifies the name of the log file where all searchd run time events will be logged. If not specified, the default name is 'searchd.log'.

Alternatively, you can use the 'syslog' as the file name. In this case, the events will be sent to the syslog daemon. To use the syslog option, you need to configure Manticore with the -–with-syslog option during building.

Limits the amount of queries per batch. Optional, default is 32.

Makes searchd perform a sanity check of the amount of queries submitted in a single batch when using multi-queries. Set it to 0 to skip the check.

Maximum number of simultaneous client connections. Unlimited by default. That is usually noticeable only when using any kind of persistent connections, like cli mysql sessions or persistent remote connections from remote distributed tables. When the limit is exceeded you can still connect to the server using the VIP connection. VIP connections are not counted towards the limit.

Instance-wide limit of threads one operation can use. By default, appropriate operations can occupy all CPU cores, leaving no room for other operations. For example, call pq against a considerably large percolate table can utilize all threads for tens of seconds. Setting max_threads_per_query to, say, half of threads will ensure that you can run a couple of such call pq operations in parallel.

You can also set this setting as a session or a global variable during runtime.

Additionally, you can control the behavior on a per-query basis with the help of the threads OPTION.

Maximum allowed per-query filter count. This setting is only used for internal sanity checks and does not directly affect RAM usage or performance. Optional, the default is 256.

Maximum allowed per-filter values count. This setting is only used for internal sanity checks and does not directly affect RAM usage or performance. Optional, the default is 4096.

The maximum number of files that the server is allowed to open is called the "soft limit". Note that serving large fragmented real-time tables may require this limit to be set high, as each disk chunk may occupy a dozen or more files. For example, a real-time table with 1000 chunks may require thousands of files to be opened simultaneously. If you encounter the error 'Too many open files' in the logs, try adjusting this option, as it may help resolve the issue.

There is also a "hard limit" that cannot be exceeded by the option. This limit is defined by the system and can be changed in the file /etc/security/limits.conf on Linux. Other operating systems may have different approaches, so consult your manuals for more information.

Apart from direct numeric values, you can use the magic word 'max' to set the limit equal to the available current hard limit.

Maximum allowed network packet size. This setting limits both query packets from clients and response packets from remote agents in a distributed environment. Only used for internal sanity checks, it does not directly affect RAM usage or performance. Optional, the default is 128M.

A server version string to return via the MySQL protocol. Optional, the default is empty (returns the Manticore version).

Several picky MySQL client libraries depend on a particular version number format used by MySQL, and moreover, sometimes choose a different execution path based on the reported version number (rather than the indicated capabilities flags). For instance, Python MySQLdb 1.2.2 throws an exception when the version number is not in X.Y.ZZ format; MySQL .NET connector 6.3.x fails internally on version numbers 1.x along with a certain combination of flags, etc. To work around that, you can use the mysql_version_string directive and have searchd report a different version to clients connecting over the MySQL protocol. (By default, it reports its own version.)

Network client request read/write timeout, in seconds (or special_suffixes). Optional, the default is 5 seconds. searchd will forcibly close a client connection which fails to send a query or read a result within this timeout.

Note also the reset_network_timeout_on_packet parameter. This parameter alters the behavior of network_timeout from applying to the entire query or result to individual packets instead. Typically, a query/result fits within one or two packets. However, in cases where a large amount of data is required, this parameter can be invaluable in maintaining active operations.

This setting allows you to specify the network address of the node. By default, it is set to the replication listen ddress. This is correct in most cases; however, there are situations where you have to specify it manually:

• Node behind a firewall
• Network address translation enabled (NAT)
• Container deployments, such as Docker or cloud deployments
• Clusters with nodes in more than one region

This setting determines whether to allow queries with only the negation full-text operator. Optional, the default is 0 (fail queries with only the NOT operator).

Sets the default table compaction threshold. Read more here - Number of optimized disk chunks. This setting can be overridden with the per-query option cutoff. It can also be changed dynamically via SET GLOBAL.

This setting determines the maximum number of simultaneous persistent connections to remote persistent agents. Each time an agent defined under agent_persistent is connected, we try to reuse an existing connection (if any), or connect and save the connection for future use. However, in some cases, it makes sense to limit the number of such persistent connections. This directive defines the limit. It affects the number of connections to each agent's host across all distributed tables.

It is reasonable to set the value equal to or less than the max_connections option in the agent's config.

pid_file is a mandatory configuration option in Manticore search that specifies the path of the file where the process ID of the searchd server is stored.

The searchd process ID file is re-created and locked on startup, and contains the head server process ID while the server is running. It is unlinked on server shutdown. The purpose of this file is to enable Manticore to perform various internal tasks, such as checking whether there is already a running instance of searchd, stopping searchd, and notifying it that it should rotate the tables. The file can also be used for external automation scripts.

Costs for the query time prediction model, in nanoseconds. Optional, the default is doc=64, hit=48, skip=2048, match=64.

Terminating queries before completion based on their execution time (with the max query time setting) is a nice safety net, but it comes with an inherent drawback: indeterministic (unstable) results. That is, if you repeat the very same (complex) search query with a time limit several times, the time limit will be hit at different stages, and you will get different result sets.

The preopen_tables configuration directive specifies whether to forcibly preopen all tables on startup. The default value is 1, which means that all tables will be preopened regardless of the per-table preopen setting. If set to 0, the per-table settings can take effect, and they will default to 0.

Pre-opening tables can prevent races between search queries and rotations that can cause queries to fail occasionally. However, it also uses more file handles. In most scenarios, it is recommended to preopen tables.

Here's an example configuration:

The pseudo_sharding configuration option enables parallelization of search queries to local plain and real-time tables, regardless of whether they are queried directly or through a distributed table. This feature will automatically parallelize queries to up to the number of threads specified in searchd.threads # of threads.

Note that if your worker threads are already busy, because you have:

• high query concurrency
• physical sharding of any kind:
  • distributed table of multiple plain/real-time tables
  • real-time table consisting of too many disk chunks

then enabling pseudo_sharding may not provide any benefits and may even result in a slight decrease in throughput. If you prioritize higher throughput over lower latency, it's recommended to disable this option.

Enabled by default.

This configuration sets the maximum amount of RAM allocated for cached result sets in bytes. The default value is 16777216, which is equivalent to 16 megabytes. If the value is set to 0, the query cache is disabled. For more information about the query cache, please refer to the query cache for details.

Query log format. Optional, allowed values are plain and sphinxql, default is sphinxql.

The sphinxql mode logs valid SQL statements. The plain mode logs queries in a plain text format (mostly suitable for purely full-text use cases). This directive allows you to switch between the two formats on search server startup. The log format can also be altered on the fly, using SET GLOBAL query_log_format=sphinxql syntax. Refer to Query logging for more details.

Query log file name. Optional, default is empty (do not log queries). All search queries (such as SELECT ... but not INSERT/REPLACE/UPDATE queries) will be logged in this file. The format is described in Query logging. In case of 'plain' format, you can use 'syslog' as the path to the log file. In this case, all search queries will be sent to the syslog daemon with LOG_INFO priority, prefixed with '[query]' instead of timestamp. To use the syslog option, Manticore must be configured with -–with-syslog on building.

The query_log_mode directive allows you to set a different permission for the searchd and query log files. By default, these log files are created with 600 permission, meaning that only the user under which the server runs and root users can read the log files. This directive can be handy if you want to allow other users to read the log files, for example, monitoring solutions running on non-root users.

The read_buffer_docs directive controls the per-keyword read buffer size for document lists. For every keyword occurrence in every search query, there are two associated read buffers: one for the document list and one for the hit list. This setting lets you control the document list buffer size.

A larger buffer size might increase per-query RAM use, but it could possibly decrease I/O time. It makes sense to set larger values for slow storage, but for storage capable of high IOPS, experimenting should be done in the low values area.

The default value is 256K, and the minimal value is 8K. You may also set read_buffer_docs on a per-table basis, which will override anything set on the server's config level.

The read_buffer_hits directive specifies the per-keyword read buffer size for hit lists in search queries. By default, the size is 256K and the minimum value is 8K. For every keyword occurrence in a search query, there are two associated read buffers, one for the document list and one for the hit list. Increasing the buffer size can increase per-query RAM use but decrease I/O time. For slow storage, larger buffer sizes make sense, while for storage capable of high IOPS, experimenting should be done in the low values area.

This setting can also be specified on a per-table basis using the read_buffer_hits option in read_buffer_hits which will override the server-level setting.

Unhinted read size. Optional, default is 32K, minimal 1K

When querying, some reads know in advance exactly how much data is there to be read, but some currently do not. Most prominently, hit list size is not currently known in advance. This setting lets you control how much data to read in such cases. It impacts hit list I/O time, reducing it for lists larger than unhinted read size, but raising it for smaller lists. It does not affect RAM usage because the read buffer will already be allocated. So it should not be greater than read_buffer.

Refines the behavior of networking timeouts (such as network_timeout, read_timeout, and agent_query_timeout).

When set to 0, timeouts limit the maximum time for sending the entire request/query. When set to 1 (default), timeouts limit the maximum time between network activities.

With replication, a node may need to send a large file (for example, 100GB) to another node. Assume the network can transfer data at 1GB/s, with a series of packets of 4-5MB each. To transfer the entire file, you would need 100 seconds. A default timeout of 5 seconds would only allow the transfer of 5GB before the connection is dropped. Increasing the timeout could be a workaround, but it is not scalable (for instance, the next file might be 150GB, leading to failure again). However, with the default reset_network_timeout_on_packet set to 1, the timeout is applied not to the entire transfer but to individual packets. As long as the transfer is in progress (and data is actually being received over the network during the timeout period), it is kept alive. If the transfer gets stuck, such that a timeout occurs between packets, it will be dropped.

Note that if you set up a distributed table, each node — both master and agents — should be tuned. On the master side, agent_query_timeout is affected; on agents, network_timeout is relevant.

RT tables RAM chunk flush check period, in seconds (or special_suffixes). Optional, default is 10 hours.

Actively updated RT tables that fully fit in RAM chunks can still result in ever-growing binlogs, impacting disk use and crash recovery time. With this directive, the search server performs periodic flush checks, and eligible RAM chunks can be saved, enabling consequential binlog cleanup. See Binary logging for more details.

A maximum number of I/O operations (per second) that the RT chunks merge thread is allowed to start. Optional, default is 0 (no limit).

This directive lets you throttle down the I/O impact arising from the OPTIMIZE statements. It is guaranteed that all RT optimization activities will not generate more disk IOPS (I/Os per second) than the configured limit. Limiting rt_merge_iops can reduce search performance degradation caused by merging.

A maximum size of an I/O operation that the RT chunks merge thread is allowed to start. Optional, default is 0 (no limit).

This directive lets you throttle down the I/O impact arising from the OPTIMIZE statements. I/Os larger than this limit will be broken down into two or more I/Os, which will then be accounted for as separate I/Os with regards to the rt_merge_iops limit. Thus, it is guaranteed that all optimization activities will not generate more than (rt_merge_iops * rt_merge_maxiosize) bytes of disk I/O per second.

Prevents searchd stalls while rotating tables with huge amounts of data to precache. Optional, default is 1 (enable seamless rotation). On Windows systems, seamless rotation is disabled by default.

Tables may contain some data that needs to be precached in RAM. At the moment, .spa, .spb, .spi, and .spm files are fully precached (they contain attribute data, blob attribute data, keyword table, and killed row map, respectively.) Without seamless rotate, rotating a table tries to use as little RAM as possible and works as follows:

1. New queries are temporarily rejected (with "retry" error code);
2. searchd waits for all currently running queries to finish;
3. The old table is deallocated, and its files are renamed;
4. New table files are renamed, and required RAM is allocated;
5. New table attribute and dictionary data are preloaded to RAM;
6. searchd resumes serving queries from the new table.

However, if there's a lot of attribute or dictionary data, then the preloading step could take a noticeable amount of time - up to several minutes in the case of preloading 1-5+ GB files.

With seamless rotate enabled, rotation works as follows:

1. New table RAM storage is allocated;
2. New table attribute and dictionary data are asynchronously preloaded to RAM;
3. On success, the old table is deallocated, and both tables' files are renamed;
4. On failure, the new table is deallocated;
5. At any given moment, queries are served either from the old or new table copy.

Seamless rotate comes at the cost of higher peak memory usage during the rotation (because both old and new copies of .spa/.spb/.spi/.spm data need to be in RAM while preloading the new copy). Average usage remains the same.

This option enables/disables the use of secondary indexes for search queries. It is optional, and the default is 1 (enabled). Note that you don't need to enable it for indexing as it is always enabled as long as the Manticore Columnar Library is installed. The latter is also required for using the indexes when searching. There are three modes available:

• 0: Disable the use of secondary indexes on search. They can be enabled for individual queries using analyzer hints
• 1: Enable the use of secondary indexes on search. They can be disabled for individual queries using analyzer hints
• force: Same as enable, but any errors during the loading of secondary indexes will be reported, and the whole index will not be loaded into the daemon.

Integer number that serves as a server identifier used as a seed to generate a unique short UUID for nodes that are part of a replication cluster. The server_id must be unique across the nodes of a cluster and in the range from 0 to 127. If server_id is not set, the MAC address or a random number will be used as a seed for the short UUID.

searchd --stopwait waiting time, in seconds (or special_suffixes). Optional, default is 60 seconds.

When you run searchd --stopwait your server needs to perform some activities before stopping, such as finishing queries, flushing RT RAM chunks, flushing attributes, and updating the binlog. These tasks require some time. searchd --stopwait will wait up to shutdown_time seconds for the server to finish its jobs. The suitable time depends on your table size and load.

A prefix to prepend to the local file names when generating snippets. Optional, default is the current working folder.

This prefix can be used in distributed snippets generation along with load_files or load_files_scattered options.

Note that this is a prefix and not a path! This means that if a prefix is set to "server1" and the request refers to "file23", searchd will attempt to open "server1file23" (all of that without quotes). So, if you need it to be a path, you have to include the trailing slash.

After constructing the final file path, the server unwinds all relative dirs and compares the final result with the value of snippet_file_prefix. If the result does not begin with the prefix, such a file will be rejected with an error message.

For example, if you set it to /mnt/data and someone calls snippet generation with the file ../../../etc/passwd as the source, they will get the error message:

File '/mnt/data/../../../etc/passwd' escapes '/mnt/data/' scope

instead of the content of the file.

Also, with a non-set parameter and reading /etc/passwd, it will actually read /daemon/working/folder/etc/passwd since the default for the parameter is the server's working folder.

Note also that this is a local option; it does not affect the agents in any way. So you can safely set a prefix on a master server. The requests routed to the agents will not be affected by the master's setting. They will, however, be affected by the agent's own settings.

This might be useful, for instance, when the document storage locations (whether local storage or NAS mountpoints) are inconsistent across the servers.

Path to a file where the current SQL state will be serialized.

On server startup, this file gets replayed. On eligible state changes (e.g., SET GLOBAL), this file gets rewritten automatically. This can prevent a hard-to-diagnose problem: If you load UDF functions but Manticore crashes, when it gets (automatically) restarted, your UDF and global variables will no longer be available. Using persistent state helps ensure a graceful recovery with no such surprises.

sphinxql_state cannot be used to execute arbitrary commands, such as CREATE TABLE.

Maximum time to wait between requests (in seconds, or special_suffixes) when using the SQL interface. Optional, default is 15 minutes.

Path to the SSL Certificate Authority (CA) certificate file (also known as root certificate). Optional, default is empty. When not empty, the certificate in ssl_cert should be signed by this root certificate.

The server uses the CA file to verify the signature on the certificate. The file must be in PEM format.

Path to the server's SSL certificate. Optional, default is empty.

The server uses this certificate as a self-signed public key to encrypt HTTP traffic over SSL. The file must be in PEM format.

Path to the SSL certificate key. Optional, default is empty.

The server uses this private key to encrypt HTTP traffic over SSL. The file must be in PEM format.

Max common subtree document cache size, per-query. Optional, default is 0 (disabled).

This setting limits the RAM usage of a common subtree optimizer (see multi-queries). At most, this much RAM will be spent to cache document entries for each query. Setting the limit to 0 disables the optimizer.

Max common subtree hit cache size, per-query. Optional, default is 0 (disabled).

This setting limits the RAM usage of a common subtree optimizer (see multi-queries). At most, this much RAM will be spent to cache keyword occurrences (hits) for each query. Setting the limit to 0 disables the optimizer.

Number of working threads (or, size of thread pool) for the Manticore daemon. Manticore creates this number of OS threads on start, and they perform all jobs inside the daemon, such as executing queries, creating snippets, etc. Some operations may be split into sub-tasks and executed in parallel, for example:

• Search in a real-time table
• Search in a distributed table consisting of local tables
• Percolate query call
• and others

By default, it's set to the number of CPU cores on the server. Manticore creates the threads on start and keeps them until it's stopped. Each sub-task can use one of the threads when it needs it. When the sub-task finishes, it releases the thread so another sub-task can use it.

In the case of intensive I/O type of load, it might make sense to set the value higher than the number of CPU cores.

Maximum stack size for a job (coroutine, one search query may cause multiple jobs/coroutines). Optional, default is 128K.

Each job has its own stack of 128K. When you run a query, it's checked for how much stack it requires. If the default 128K is enough, it's just processed. If it needs more, another job with an increased stack is scheduled, which continues processing. The maximum size of such an advanced stack is limited by this setting.

Setting the value to a reasonably high rate will help with processing very deep queries without implying that overall RAM consumption will grow too high. For example, setting it to 1G does not imply that every new job will take 1G of RAM, but if we see that it requires, let's say, 100M stack, we just allocate 100M for the job. Other jobs at the same time will be running with their default 128K stack. The same way, we can run even more complex queries that need 500M. And only if we see internally that the job requires more than 1G of stack, we will fail and report about too low thread_stack.

However, in practice, even a query which needs 16M of stack is often too complex for parsing and consumes too much time and resources to be processed. So, the daemon will process it, but limiting such queries by the thread_stack setting looks quite reasonable.

Determines whether to unlink .old table copies on successful rotation. Optional, default is 1 (do unlink).

Threaded server watchdog. Optional, default is 1 (watchdog enabled).

When a Manticore query crashes, it can take down the entire server. With the watchdog feature enabled, searchd also maintains a separate lightweight process that monitors the main server process and automatically restarts it in case of abnormal termination. The watchdog is enabled by default.